Data Description¶

The data provided is a transformed version of the original data which was collected using sensors.

  • Train.csv - To be used for training and tuning of models.
  • Test.csv - To be used only for testing the performance of the final best model.

Both the datasets consist of 40 predictor variables and 1 target variable.

Installing and Importing the necessary libraries¶

In [ ]:
# Installing the libraries with the specified version
#!pip install --no-deps tensorflow==2.18.0 scikit-learn==1.3.2 matplotlib===3.8.3 seaborn==0.13.2 numpy==1.26.4 pandas==2.2.2 -q --user --no-warn-script-location
In [ ]:
#Library for data manupulation and analysis
import numpy as np
#Package for scientific computation
import pandas as pd
#Library for data visualization
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import confusion_matrix
#Splitting dataset in training and test
from sklearn.model_selection import train_test_split
#Time function
import time
#import  tools for data processing
from sklearn.preprocessing import StandardScaler,LabelEncoder,OneHotEncoder
#import simplerimputer
from sklearn.impute import SimpleImputer
#Import function for evaluating performance of model
from sklearn.metrics import f1_score,accuracy_score,precision_score,recall_score,classification_report
#import metrics
from sklearn import metrics
#Imports the tensorflow,keras and layers
import tensorflow
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential # Corrected import statement
from tensorflow.keras.layers import Dense,Dropout,Input,BatchNormalization
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.utils import to_categorical
from tensorflow.keras import backend
# to suppress unnecessary warnings
import warnings
warnings.filterwarnings("ignore")
In [ ]:
# Set the seed using keras.utils.set_random_seed. This will set:
# 1) `numpy` seed
# 2) backend random seed
# 3) `python` random seed
keras.utils.set_random_seed(812)

# If using TensorFlow, this will make GPU ops as deterministic as possible,
# but it might affect the overall performance
tf.config.experimental.enable_op_determinism()

Loading the Data¶

In [ ]:
# uncomment and run the following lines for Google Colab
#from google.colab import drive
#drive.mount('/content/drive')
In [ ]:
data=pd.read_csv('Train.csv')
data_test=pd.read_csv('Test.csv')
In [ ]:
#creating copies
df=data.copy()
df_test=data_test.copy()

Data Overview¶

Understand shape of the dataset¶

In [ ]:
#shape of the Train dataset
print(f'The Train dataset has {df.shape[0]} rows and {df.shape[1]} columns ')
The Train dataset has 20000 rows and 41 columns 
In [ ]:
#shape of the Test dataset
print(f'The Test dataset has {df_test.shape[0]} rows and {df_test.shape[1]} columns ')
The Test dataset has 5000 rows and 41 columns 

View the first 5 rows of the datasets¶

In [ ]:
#First five raws of train
df.head()
Out[ ]:
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 ... V32 V33 V34 V35 V36 V37 V38 V39 V40 Target
0 -4.464606 -4.679129 3.101546 0.506130 -0.221083 -2.032511 -2.910870 0.050714 -1.522351 3.761892 ... 3.059700 -1.690440 2.846296 2.235198 6.667486 0.443809 -2.369169 2.950578 -3.480324 0
1 3.365912 3.653381 0.909671 -1.367528 0.332016 2.358938 0.732600 -4.332135 0.565695 -0.101080 ... -1.795474 3.032780 -2.467514 1.894599 -2.297780 -1.731048 5.908837 -0.386345 0.616242 0
2 -3.831843 -5.824444 0.634031 -2.418815 -1.773827 1.016824 -2.098941 -3.173204 -2.081860 5.392621 ... -0.257101 0.803550 4.086219 2.292138 5.360850 0.351993 2.940021 3.839160 -4.309402 0
3 1.618098 1.888342 7.046143 -1.147285 0.083080 -1.529780 0.207309 -2.493629 0.344926 2.118578 ... -3.584425 -2.577474 1.363769 0.622714 5.550100 -1.526796 0.138853 3.101430 -1.277378 0
4 -0.111440 3.872488 -3.758361 -2.982897 3.792714 0.544960 0.205433 4.848994 -1.854920 -6.220023 ... 8.265896 6.629213 -10.068689 1.222987 -3.229763 1.686909 -2.163896 -3.644622 6.510338 0

5 rows × 41 columns

In [ ]:
#First five raws of test
df_test.head()
Out[ ]:
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 ... V32 V33 V34 V35 V36 V37 V38 V39 V40 Target
0 -0.613489 -3.819640 2.202302 1.300420 -1.184929 -4.495964 -1.835817 4.722989 1.206140 -0.341909 ... 2.291204 -5.411388 0.870073 0.574479 4.157191 1.428093 -10.511342 0.454664 -1.448363 0
1 0.389608 -0.512341 0.527053 -2.576776 -1.016766 2.235112 -0.441301 -4.405744 -0.332869 1.966794 ... -2.474936 2.493582 0.315165 2.059288 0.683859 -0.485452 5.128350 1.720744 -1.488235 0
2 -0.874861 -0.640632 4.084202 -1.590454 0.525855 -1.957592 -0.695367 1.347309 -1.732348 0.466500 ... -1.318888 -2.997464 0.459664 0.619774 5.631504 1.323512 -1.752154 1.808302 1.675748 0
3 0.238384 1.458607 4.014528 2.534478 1.196987 -3.117330 -0.924035 0.269493 1.322436 0.702345 ... 3.517918 -3.074085 -0.284220 0.954576 3.029331 -1.367198 -3.412140 0.906000 -2.450889 0
4 5.828225 2.768260 -1.234530 2.809264 -1.641648 -1.406698 0.568643 0.965043 1.918379 -2.774855 ... 1.773841 -1.501573 -2.226702 4.776830 -6.559698 -0.805551 -0.276007 -3.858207 -0.537694 0

5 rows × 41 columns

check the datatypes of column of the datasets¶

In [ ]:
#datatype for train
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 20000 entries, 0 to 19999
Data columns (total 41 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   V1      19982 non-null  float64
 1   V2      19982 non-null  float64
 2   V3      20000 non-null  float64
 3   V4      20000 non-null  float64
 4   V5      20000 non-null  float64
 5   V6      20000 non-null  float64
 6   V7      20000 non-null  float64
 7   V8      20000 non-null  float64
 8   V9      20000 non-null  float64
 9   V10     20000 non-null  float64
 10  V11     20000 non-null  float64
 11  V12     20000 non-null  float64
 12  V13     20000 non-null  float64
 13  V14     20000 non-null  float64
 14  V15     20000 non-null  float64
 15  V16     20000 non-null  float64
 16  V17     20000 non-null  float64
 17  V18     20000 non-null  float64
 18  V19     20000 non-null  float64
 19  V20     20000 non-null  float64
 20  V21     20000 non-null  float64
 21  V22     20000 non-null  float64
 22  V23     20000 non-null  float64
 23  V24     20000 non-null  float64
 24  V25     20000 non-null  float64
 25  V26     20000 non-null  float64
 26  V27     20000 non-null  float64
 27  V28     20000 non-null  float64
 28  V29     20000 non-null  float64
 29  V30     20000 non-null  float64
 30  V31     20000 non-null  float64
 31  V32     20000 non-null  float64
 32  V33     20000 non-null  float64
 33  V34     20000 non-null  float64
 34  V35     20000 non-null  float64
 35  V36     20000 non-null  float64
 36  V37     20000 non-null  float64
 37  V38     20000 non-null  float64
 38  V39     20000 non-null  float64
 39  V40     20000 non-null  float64
 40  Target  20000 non-null  int64  
dtypes: float64(40), int64(1)
memory usage: 6.3 MB
In [ ]:
#datatype for test
df_test.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5000 entries, 0 to 4999
Data columns (total 41 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   V1      4995 non-null   float64
 1   V2      4994 non-null   float64
 2   V3      5000 non-null   float64
 3   V4      5000 non-null   float64
 4   V5      5000 non-null   float64
 5   V6      5000 non-null   float64
 6   V7      5000 non-null   float64
 7   V8      5000 non-null   float64
 8   V9      5000 non-null   float64
 9   V10     5000 non-null   float64
 10  V11     5000 non-null   float64
 11  V12     5000 non-null   float64
 12  V13     5000 non-null   float64
 13  V14     5000 non-null   float64
 14  V15     5000 non-null   float64
 15  V16     5000 non-null   float64
 16  V17     5000 non-null   float64
 17  V18     5000 non-null   float64
 18  V19     5000 non-null   float64
 19  V20     5000 non-null   float64
 20  V21     5000 non-null   float64
 21  V22     5000 non-null   float64
 22  V23     5000 non-null   float64
 23  V24     5000 non-null   float64
 24  V25     5000 non-null   float64
 25  V26     5000 non-null   float64
 26  V27     5000 non-null   float64
 27  V28     5000 non-null   float64
 28  V29     5000 non-null   float64
 29  V30     5000 non-null   float64
 30  V31     5000 non-null   float64
 31  V32     5000 non-null   float64
 32  V33     5000 non-null   float64
 33  V34     5000 non-null   float64
 34  V35     5000 non-null   float64
 35  V36     5000 non-null   float64
 36  V37     5000 non-null   float64
 37  V38     5000 non-null   float64
 38  V39     5000 non-null   float64
 39  V40     5000 non-null   float64
 40  Target  5000 non-null   int64  
dtypes: float64(40), int64(1)
memory usage: 1.6 MB

In both datasets Columns V1 to V40 are float while target is integer and there are missing values in V1 and V2

Checking for missing values¶

In [ ]:
#Checking for missing values in train
df.isnull().sum()
Out[ ]:
0
V1 18
V2 18
V3 0
V4 0
V5 0
V6 0
V7 0
V8 0
V9 0
V10 0
V11 0
V12 0
V13 0
V14 0
V15 0
V16 0
V17 0
V18 0
V19 0
V20 0
V21 0
V22 0
V23 0
V24 0
V25 0
V26 0
V27 0
V28 0
V29 0
V30 0
V31 0
V32 0
V33 0
V34 0
V35 0
V36 0
V37 0
V38 0
V39 0
V40 0
Target 0

In [ ]:
#Checking for missing values in test
df_test.isnull().sum()
Out[ ]:
0
V1 5
V2 6
V3 0
V4 0
V5 0
V6 0
V7 0
V8 0
V9 0
V10 0
V11 0
V12 0
V13 0
V14 0
V15 0
V16 0
V17 0
V18 0
V19 0
V20 0
V21 0
V22 0
V23 0
V24 0
V25 0
V26 0
V27 0
V28 0
V29 0
V30 0
V31 0
V32 0
V33 0
V34 0
V35 0
V36 0
V37 0
V38 0
V39 0
V40 0
Target 0

There are missing values for V1 and V2 in both datasets

Checking for duplicates¶

In [ ]:
#Duplicates in train
df.duplicated().sum()
Out[ ]:
np.int64(0)

There are no duplicates in the train dataset

Let's check the statistical summary of the data¶

In [ ]:
#statistical summary of train
df.describe().T
Out[ ]:
count mean std min 25% 50% 75% max
V1 19982.0 -0.271996 3.441625 -11.876451 -2.737146 -0.747917 1.840112 15.493002
V2 19982.0 0.440430 3.150784 -12.319951 -1.640674 0.471536 2.543967 13.089269
V3 20000.0 2.484699 3.388963 -10.708139 0.206860 2.255786 4.566165 17.090919
V4 20000.0 -0.083152 3.431595 -15.082052 -2.347660 -0.135241 2.130615 13.236381
V5 20000.0 -0.053752 2.104801 -8.603361 -1.535607 -0.101952 1.340480 8.133797
V6 20000.0 -0.995443 2.040970 -10.227147 -2.347238 -1.000515 0.380330 6.975847
V7 20000.0 -0.879325 1.761626 -7.949681 -2.030926 -0.917179 0.223695 8.006091
V8 20000.0 -0.548195 3.295756 -15.657561 -2.642665 -0.389085 1.722965 11.679495
V9 20000.0 -0.016808 2.160568 -8.596313 -1.494973 -0.067597 1.409203 8.137580
V10 20000.0 -0.012998 2.193201 -9.853957 -1.411212 0.100973 1.477045 8.108472
V11 20000.0 -1.895393 3.124322 -14.832058 -3.922404 -1.921237 0.118906 11.826433
V12 20000.0 1.604825 2.930454 -12.948007 -0.396514 1.507841 3.571454 15.080698
V13 20000.0 1.580486 2.874658 -13.228247 -0.223545 1.637185 3.459886 15.419616
V14 20000.0 -0.950632 1.789651 -7.738593 -2.170741 -0.957163 0.270677 5.670664
V15 20000.0 -2.414993 3.354974 -16.416606 -4.415322 -2.382617 -0.359052 12.246455
V16 20000.0 -2.925225 4.221717 -20.374158 -5.634240 -2.682705 -0.095046 13.583212
V17 20000.0 -0.134261 3.345462 -14.091184 -2.215611 -0.014580 2.068751 16.756432
V18 20000.0 1.189347 2.592276 -11.643994 -0.403917 0.883398 2.571770 13.179863
V19 20000.0 1.181808 3.396925 -13.491784 -1.050168 1.279061 3.493299 13.237742
V20 20000.0 0.023608 3.669477 -13.922659 -2.432953 0.033415 2.512372 16.052339
V21 20000.0 -3.611252 3.567690 -17.956231 -5.930360 -3.532888 -1.265884 13.840473
V22 20000.0 0.951835 1.651547 -10.122095 -0.118127 0.974687 2.025594 7.409856
V23 20000.0 -0.366116 4.031860 -14.866128 -3.098756 -0.262093 2.451750 14.458734
V24 20000.0 1.134389 3.912069 -16.387147 -1.468062 0.969048 3.545975 17.163291
V25 20000.0 -0.002186 2.016740 -8.228266 -1.365178 0.025050 1.397112 8.223389
V26 20000.0 1.873785 3.435137 -11.834271 -0.337863 1.950531 4.130037 16.836410
V27 20000.0 -0.612413 4.368847 -14.904939 -3.652323 -0.884894 2.189177 17.560404
V28 20000.0 -0.883218 1.917713 -9.269489 -2.171218 -0.891073 0.375884 6.527643
V29 20000.0 -0.985625 2.684365 -12.579469 -2.787443 -1.176181 0.629773 10.722055
V30 20000.0 -0.015534 3.005258 -14.796047 -1.867114 0.184346 2.036229 12.505812
V31 20000.0 0.486842 3.461384 -13.722760 -1.817772 0.490304 2.730688 17.255090
V32 20000.0 0.303799 5.500400 -19.876502 -3.420469 0.052073 3.761722 23.633187
V33 20000.0 0.049825 3.575285 -16.898353 -2.242857 -0.066249 2.255134 16.692486
V34 20000.0 -0.462702 3.183841 -17.985094 -2.136984 -0.255008 1.436935 14.358213
V35 20000.0 2.229620 2.937102 -15.349803 0.336191 2.098633 4.064358 15.291065
V36 20000.0 1.514809 3.800860 -14.833178 -0.943809 1.566526 3.983939 19.329576
V37 20000.0 0.011316 1.788165 -5.478350 -1.255819 -0.128435 1.175533 7.467006
V38 20000.0 -0.344025 3.948147 -17.375002 -2.987638 -0.316849 2.279399 15.289923
V39 20000.0 0.890653 1.753054 -6.438880 -0.272250 0.919261 2.057540 7.759877
V40 20000.0 -0.875630 3.012155 -11.023935 -2.940193 -0.920806 1.119897 10.654265
Target 20000.0 0.055500 0.228959 0.000000 0.000000 0.000000 0.000000 1.000000

Exploratory Data Analysis¶

Univariate analysis¶

In [ ]:
def histogram_boxplot(data, feature, figsize=(15, 10), kde=False, bins=None):
    """
    Boxplot and histogram combined

    data: dataframe
    feature: dataframe column
    figsize: size of figure (default (15,10))
    kde: whether to show the density curve (default False)
    bins: number of bins for histogram (default None)
    """
    f2, (ax_box2, ax_hist2) = plt.subplots(
        nrows=2,  # Number of rows of the subplot grid= 2
        sharex=True,  # x-axis will be shared among all subplots
        gridspec_kw={"height_ratios": (0.25, 0.75)},
        figsize=figsize,
    )  # creating the 2 subplots
    sns.boxplot(
        data=data, x=feature, ax=ax_box2, showmeans=True, color="violet"
    )  # boxplot will be created and a triangle will indicate the mean value of the column
    sns.histplot(
        data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins
    ) if bins else sns.histplot(
        data=data, x=feature, kde=kde, ax=ax_hist2
    )  # For histogram
    ax_hist2.axvline(
        data[feature].mean(), color="green", linestyle="--"
    )  # Add mean to the histogram
    ax_hist2.axvline(
        data[feature].median(), color="black", linestyle="-"
    )  # Add median to the histogram

For Features in Train dataset

In [ ]:
cols_list = df.select_dtypes(include=np.number).columns.tolist()
cols_list.remove("Target")
for feature in cols_list:
    histogram_boxplot(df, feature)
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

The features in the dataset V1 to V40 are almost symentrical in distribution.

Distribution of target¶

In [ ]:
#Distribution in Train
df.Target.value_counts(normalize=True)
Out[ ]:
proportion
Target
0 0.9445
1 0.0555

In [ ]:
#Distribution in test
df_test.Target.value_counts(normalize=True)
Out[ ]:
proportion
Target
0 0.9436
1 0.0564

The dataset is imbalance with no failure having significantly fewer instances

Bivariate Analysis¶

In [ ]:
#Plot correlation between features
plt.figure(figsize=(30,20))
sns.heatmap(df[cols_list].corr(),annot=True,vmin=-1,vmax=1,cmap='BrBG')
plt.show()
No description has been provided for this image

The highest negative correlation is between V14 and V2 at 0.85 while highest positive correlation is between V15 and V7 at 0.87.

Creating functions that will help us with further analysis.¶

In [ ]:
### function to plot distributions wrt target


def distribution_plot_wrt_target(data, predictor, target):

    fig, axs = plt.subplots(2, 2, figsize=(12, 10))

    target_uniq = data[target].unique()

    axs[0, 0].set_title("Distribution of target for target=" + str(target_uniq[0]))
    sns.histplot(
        data=data[data[target] == target_uniq[0]],
        x=predictor,
        kde=True,
        ax=axs[0, 0],
        color="teal",
        stat="density",
    )

    axs[0, 1].set_title("Distribution of target for target=" + str(target_uniq[1]))
    sns.histplot(
        data=data[data[target] == target_uniq[1]],
        x=predictor,
        kde=True,
        ax=axs[0, 1],
        color="orange",
        stat="density",
    )

    axs[1, 0].set_title("Boxplot w.r.t target")
    sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette="gist_rainbow")

    axs[1, 1].set_title("Boxplot (without outliers) w.r.t target")
    sns.boxplot(
        data=data,
        x=target,
        y=predictor,
        ax=axs[1, 1],
        showfliers=False,
        palette="gist_rainbow",
    )

    plt.tight_layout()
    plt.show()
In [ ]:
for feature in cols_list:
    distribution_plot_wrt_target(df, feature, 'Target')
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

There is no variation in distribution between features V2,V6,V12,V25,V27,V30 and V37 with Target. While the other features display Variation with Target possible indication of impact.

Data Preprocessing¶

Separating response variable and predictors in train dataset¶

In [ ]:
X=df.drop('Target',axis=1)
y=df['Target']
In [ ]:
#printing the shape of the data
print(X.shape)
print(y.shape)
(20000, 40)
(20000,)

Splitting the train dataset into train and Validation dataset¶

In [ ]:
X_train,X_val,y_train,y_val=train_test_split(X,y,test_size=0.2,random_state=1,stratify=y)
In [ ]:
#checking the shape of dataset
print(X_train.shape)
print(X_val.shape)
print(y_train.shape)
print(y_val.shape)
(16000, 40)
(4000, 40)
(16000,)
(4000,)

Separating response variable and predictors in test dataset¶

In [ ]:
X_test=df_test.drop('Target',axis=1)
y_test=df_test['Target']
In [ ]:
print(X_test.shape)
print(y_test.shape)
(5000, 40)
(5000,)

Imputing missing values¶

V1 and V2 had missing values we will impute the missing values using median.

In [ ]:
#defining SimpleImputer
imputer=SimpleImputer(strategy='median')
In [ ]:
#Fit and transform training data
X_train=pd.DataFrame(imputer.fit_transform(X_train),columns=X_train.columns)
#Transform validation data
X_val=pd.DataFrame(imputer.transform(X_val),columns=X_train.columns)
#Transform test data
X_test=pd.DataFrame(imputer.transform(X_test),columns=X_train.columns)
In [ ]:
#checking missing values
print(X_train.isnull().sum().sum())
print(X_val.isnull().sum().sum())
print(X_test.isnull().sum().sum())
0
0
0

Model Building¶

Model Evaluation Criterion¶

The model will make predictions as follows:

  • True positives (TP) are failures correctly predicted by the model. These will result in repairing costs.
  • False negatives (FN) are real failures where there is no detection by the model. These will result in replacement costs.
  • False positives (FP) are detections where there is no failure. These will result in inspection costs.

Since it is given that the cost of repairing a generator is much less than the cost of replacing it, and the cost of inspection is less than the cost of repair hence,the company might be interested most in minimising cost of replacement by reducing FN which can be achieved by Maximising Recall

As we have are dealing with an imbalance in class distribution, we will be using class weights to allow the model to give proportionally more importance to the minority class.

In [ ]:
# Calculate class weights for imbalanced dataset
cw = (y_train.shape[0]) / np.bincount(y_train)

# Create a dictionary mapping class indices to their respective class weights
cw_dict = {}
for i in range(cw.shape[0]):
    cw_dict[i] = cw[i]

cw_dict
Out[ ]:
{0: np.float64(1.0587612493382743), 1: np.float64(18.01801801801802)}

Utility functions¶

In [ ]:
def plot(history, name):
    """
    Function to plot loss/accuracy

    history: an object which stores the metrics and losses.
    name: can be one of Loss or Accuracy
    """
    #Creating a subplot with figure and axes.
    fig, ax = plt.subplots()
    #Plotting the train accuracy or train loss
    plt.plot(history.history[name])
    #Plotting the validation accuracy or validation loss
    plt.plot(history.history['val_'+name])
    #Defining the title of the plot.
    plt.title('Model ' + name.capitalize())
    #Capitalizing the first letter.
    plt.ylabel(name.capitalize())
    #Defining the label for the x-axis.
    plt.xlabel('Epoch')
    #Defining the legend, loc controls the position of the legend.
    fig.legend(['Train', 'Validation'], loc="outside right upper")

dataframe to store the results from all the models we build¶

In [ ]:
# defining a function to compute different metrics to check performance of a classification model built using statsmodels
def model_performance_classification(
    model, predictors, target, threshold=0.5
):
    """
    Function to compute different metrics to check classification model performance

    model: classifier
    predictors: independent variables
    target: dependent variable
    threshold: threshold for classifying the observation as class 1
    """

    # checking which probabilities are greater than threshold
    pred = model.predict(predictors) > threshold

    # to compute Accuracy
    acc = accuracy_score(target, pred)
    # to compute Recall
    recall = recall_score(target, pred, average='weighted')
    # to compute Precision
    precision = precision_score(target, pred, average='weighted')
    # to compute F1-score
    f1 = f1_score(target, pred, average='weighted')

    # creating a dataframe of metrics
    df_perf = pd.DataFrame(
        {"Accuracy": acc, "Recall": recall, "Precision": precision, "F1 Score": f1,},
        index=[0],
    )

    return df_perf

Initial Model Building (Model 0)¶

  • Let's start with a neural network consisting of
    • just one hidden layer
    • activation function of ReLU
    • SGD as the optimizer
In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_0 = Sequential()
model_0.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_0.add(Dense(32,activation="relu"))
model_0.add(Dense(1,activation = 'sigmoid'))
#compiling the model With SDG as optimiser
optimizer=keras.optimizers.SGD()
model_0.compile(optimizer = optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_0.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.7752 - recall: 0.8306 - val_loss: 0.1977 - val_recall: 0.8784
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.4029 - recall: 0.9043 - val_loss: 0.1678 - val_recall: 0.8874
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3533 - recall: 0.9128 - val_loss: 0.1621 - val_recall: 0.8874
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3282 - recall: 0.9154 - val_loss: 0.1571 - val_recall: 0.8964
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3092 - recall: 0.9151 - val_loss: 0.1509 - val_recall: 0.8964
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2935 - recall: 0.9169 - val_loss: 0.1526 - val_recall: 0.8964
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2805 - recall: 0.9229 - val_loss: 0.1490 - val_recall: 0.9009
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2688 - recall: 0.9239 - val_loss: 0.1413 - val_recall: 0.8964
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2600 - recall: 0.9268 - val_loss: 0.1406 - val_recall: 0.8964
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2516 - recall: 0.9235 - val_loss: 0.1320 - val_recall: 0.8919
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2394 - recall: 0.9283 - val_loss: 0.1305 - val_recall: 0.8919
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2364 - recall: 0.9276 - val_loss: 0.1269 - val_recall: 0.8964
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2307 - recall: 0.9262 - val_loss: 0.1292 - val_recall: 0.8964
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2222 - recall: 0.9312 - val_loss: 0.1287 - val_recall: 0.8964
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2169 - recall: 0.9345 - val_loss: 0.1241 - val_recall: 0.8964
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2113 - recall: 0.9346 - val_loss: 0.1278 - val_recall: 0.8964
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2050 - recall: 0.9321 - val_loss: 0.1218 - val_recall: 0.8964
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1989 - recall: 0.9364 - val_loss: 0.1162 - val_recall: 0.8919
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1991 - recall: 0.9341 - val_loss: 0.1185 - val_recall: 0.8919
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1910 - recall: 0.9348 - val_loss: 0.1173 - val_recall: 0.8919
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1849 - recall: 0.9361 - val_loss: 0.1266 - val_recall: 0.8919
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1792 - recall: 0.9354 - val_loss: 0.1149 - val_recall: 0.8919
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1748 - recall: 0.9387 - val_loss: 0.1168 - val_recall: 0.8874
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1777 - recall: 0.9390 - val_loss: 0.1129 - val_recall: 0.8874
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.1718 - recall: 0.9430 - val_loss: 0.1184 - val_recall: 0.8964
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1693 - recall: 0.9380 - val_loss: 0.1212 - val_recall: 0.8919
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1673 - recall: 0.9401 - val_loss: 0.1171 - val_recall: 0.8829
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1642 - recall: 0.9462 - val_loss: 0.1193 - val_recall: 0.8874
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1463 - recall: 0.9449 - val_loss: 0.1223 - val_recall: 0.8874
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1685 - recall: 0.9389 - val_loss: 0.1120 - val_recall: 0.8874
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  32.86089372634888
In [ ]:
model_0.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 4,739 (18.52 KB)
 Trainable params: 4,737 (18.50 KB)
 Non-trainable params: 0 (0.00 B)
 Optimizer params: 2 (12.00 B)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_0_train_perf=model_performance_classification(model_0,X_train,y_train)
model_0_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 709us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.98175 0.98175 0.984608 0.982628
In [ ]:
model_0_val_perf=model_performance_classification(model_0,X_val,y_val)
model_0_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step  
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.978 0.978 0.980238 0.978812

Both recall for test and train are around 98% percent though wide gap between validation loss function not smooth but oscilating.

Model Performance Improvement¶

Model 1¶

In model 1 we add amomentum to check whether it's accelerating the learning process

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_1 = Sequential()
model_1.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_1.add(Dense(32,activation="relu"))
model_1.add(Dense(1,activation = 'sigmoid'))
#compiling the model With SDG with momentum as optimiser
optimizer =keras.optimizers.SGD(momentum=0.9)
model_1.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_1.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.7851 - recall: 0.8329 - val_loss: 0.2611 - val_recall: 0.8919
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.3576 - recall: 0.9007 - val_loss: 0.2256 - val_recall: 0.9054
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3194 - recall: 0.9109 - val_loss: 0.1662 - val_recall: 0.8964
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3158 - recall: 0.9217 - val_loss: 0.1611 - val_recall: 0.8829
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2944 - recall: 0.9186 - val_loss: 0.1425 - val_recall: 0.8694
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2862 - recall: 0.9219 - val_loss: 0.1462 - val_recall: 0.8874
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2694 - recall: 0.9239 - val_loss: 0.1583 - val_recall: 0.8919
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2779 - recall: 0.9220 - val_loss: 0.1526 - val_recall: 0.8919
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2605 - recall: 0.9287 - val_loss: 0.1419 - val_recall: 0.8784
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2522 - recall: 0.9317 - val_loss: 0.1548 - val_recall: 0.8829
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2517 - recall: 0.9310 - val_loss: 0.1554 - val_recall: 0.8784
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2498 - recall: 0.9317 - val_loss: 0.1703 - val_recall: 0.8829
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2430 - recall: 0.9317 - val_loss: 0.1766 - val_recall: 0.8964
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2397 - recall: 0.9350 - val_loss: 0.1976 - val_recall: 0.8919
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.2573 - recall: 0.9277 - val_loss: 0.1822 - val_recall: 0.8964
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2624 - recall: 0.9342 - val_loss: 0.3325 - val_recall: 0.8919
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2957 - recall: 0.9233 - val_loss: 0.1703 - val_recall: 0.8874
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2483 - recall: 0.9317 - val_loss: 0.2299 - val_recall: 0.8829
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2435 - recall: 0.9343 - val_loss: 0.2095 - val_recall: 0.8784
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2363 - recall: 0.9364 - val_loss: 0.2124 - val_recall: 0.8829
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2370 - recall: 0.9364 - val_loss: 0.2471 - val_recall: 0.8874
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2423 - recall: 0.9373 - val_loss: 0.2205 - val_recall: 0.8874
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2358 - recall: 0.9371 - val_loss: 0.2316 - val_recall: 0.8784
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2812 - recall: 0.9287 - val_loss: 0.2032 - val_recall: 0.8919
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.2538 - recall: 0.9334 - val_loss: 0.2751 - val_recall: 0.8829
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2772 - recall: 0.9282 - val_loss: 0.2316 - val_recall: 0.8919
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2665 - recall: 0.9299 - val_loss: 0.1797 - val_recall: 0.8874
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2458 - recall: 0.9328 - val_loss: 0.2056 - val_recall: 0.8874
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2407 - recall: 0.9363 - val_loss: 0.1574 - val_recall: 0.8784
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2325 - recall: 0.9370 - val_loss: 0.1633 - val_recall: 0.8874
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  37.33909344673157
In [ ]:
model_1.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 9,476 (37.02 KB)
 Trainable params: 4,737 (18.50 KB)
 Non-trainable params: 0 (0.00 B)
 Optimizer params: 4,739 (18.52 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_1_train_perf=model_performance_classification(model_1,X_train,y_train)
model_1_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 675us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.993437 0.993437 0.993372 0.993393
In [ ]:
model_1_val_perf=model_performance_classification(model_1,X_val,y_val)
model_1_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 745us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.988 0.988 0.987951 0.987974

Both recall in test and train have improved to 99%. The loss plots are non-smooth and oscilating.

Model 2¶

Using Adam optimizer with no regularization

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_2 = Sequential()
model_2.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_2.add(Dense(32,activation="relu"))
model_2.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam()
model_2.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_2.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.8588 - recall: 0.7826 - val_loss: 0.2708 - val_recall: 0.9054
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3898 - recall: 0.9018 - val_loss: 0.2371 - val_recall: 0.9054
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3387 - recall: 0.9118 - val_loss: 0.2323 - val_recall: 0.9009
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3120 - recall: 0.9170 - val_loss: 0.2348 - val_recall: 0.8964
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2956 - recall: 0.9198 - val_loss: 0.2017 - val_recall: 0.8964
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2782 - recall: 0.9208 - val_loss: 0.1886 - val_recall: 0.9009
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2609 - recall: 0.9224 - val_loss: 0.1766 - val_recall: 0.8964
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2499 - recall: 0.9265 - val_loss: 0.1485 - val_recall: 0.8919
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.2417 - recall: 0.9298 - val_loss: 0.1586 - val_recall: 0.8919
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2285 - recall: 0.9304 - val_loss: 0.1449 - val_recall: 0.8829
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2239 - recall: 0.9336 - val_loss: 0.1443 - val_recall: 0.8919
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2147 - recall: 0.9332 - val_loss: 0.1328 - val_recall: 0.8874
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.2037 - recall: 0.9355 - val_loss: 0.1361 - val_recall: 0.8874
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1972 - recall: 0.9375 - val_loss: 0.1543 - val_recall: 0.8919
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1915 - recall: 0.9377 - val_loss: 0.1375 - val_recall: 0.8829
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1841 - recall: 0.9394 - val_loss: 0.1317 - val_recall: 0.8784
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1797 - recall: 0.9436 - val_loss: 0.1608 - val_recall: 0.8919
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1746 - recall: 0.9492 - val_loss: 0.1275 - val_recall: 0.8784
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.1639 - recall: 0.9464 - val_loss: 0.1397 - val_recall: 0.8874
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1587 - recall: 0.9514 - val_loss: 0.1407 - val_recall: 0.8829
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1495 - recall: 0.9498 - val_loss: 0.1272 - val_recall: 0.8784
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1436 - recall: 0.9519 - val_loss: 0.1369 - val_recall: 0.8784
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1457 - recall: 0.9530 - val_loss: 0.1292 - val_recall: 0.8829
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1335 - recall: 0.9560 - val_loss: 0.1585 - val_recall: 0.8874
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1410 - recall: 0.9564 - val_loss: 0.1296 - val_recall: 0.8739
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1282 - recall: 0.9552 - val_loss: 0.1201 - val_recall: 0.8784
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1135 - recall: 0.9624 - val_loss: 0.1183 - val_recall: 0.8784
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1143 - recall: 0.9609 - val_loss: 0.1246 - val_recall: 0.8784
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.1140 - recall: 0.9599 - val_loss: 0.1302 - val_recall: 0.8784
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.1084 - recall: 0.9633 - val_loss: 0.1190 - val_recall: 0.8784
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  33.61634564399719
In [ ]:
model_2.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 14,213 (55.52 KB)
 Trainable params: 4,737 (18.50 KB)
 Non-trainable params: 0 (0.00 B)
 Optimizer params: 9,476 (37.02 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_2_train_perf=model_performance_classification(model_2,X_train,y_train)
model_2_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 705us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.987812 0.987812 0.989453 0.988276
In [ ]:
model_2_val_perf=model_performance_classification(model_2,X_val,y_val)
model_2_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.97925 0.97925 0.980815 0.979846

Model perfomance in terms of recall has reduced slightly in both cases though the loss plot are smoothening with slight oscilation

Model 3¶

using Adam optimizer with dropout regularization

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_3 = Sequential()
model_3.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_3.add(Dropout(0.4))
model_3.add(Dense(32,activation="relu"))
model_3.add(Dropout(0.2))
model_3.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam optimiser
optimizer = keras.optimizers.Adam()
model_3.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_3.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.3322 - recall: 0.6637 - val_loss: 0.3584 - val_recall: 0.8919
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6297 - recall: 0.8494 - val_loss: 0.3284 - val_recall: 0.8919
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5644 - recall: 0.8693 - val_loss: 0.2597 - val_recall: 0.8784
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4908 - recall: 0.8738 - val_loss: 0.2307 - val_recall: 0.8874
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4413 - recall: 0.8994 - val_loss: 0.2294 - val_recall: 0.8874
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4386 - recall: 0.8874 - val_loss: 0.2230 - val_recall: 0.8964
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4308 - recall: 0.8996 - val_loss: 0.2273 - val_recall: 0.8964
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3995 - recall: 0.8941 - val_loss: 0.2143 - val_recall: 0.8964
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4082 - recall: 0.9002 - val_loss: 0.1864 - val_recall: 0.8964
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4184 - recall: 0.8849 - val_loss: 0.1963 - val_recall: 0.8919
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3834 - recall: 0.8952 - val_loss: 0.1848 - val_recall: 0.8964
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3765 - recall: 0.9043 - val_loss: 0.1827 - val_recall: 0.8874
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3778 - recall: 0.8941 - val_loss: 0.1703 - val_recall: 0.8964
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3544 - recall: 0.9012 - val_loss: 0.1776 - val_recall: 0.8919
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3466 - recall: 0.9055 - val_loss: 0.1565 - val_recall: 0.8919
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3444 - recall: 0.9066 - val_loss: 0.1756 - val_recall: 0.8919
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3449 - recall: 0.9026 - val_loss: 0.1720 - val_recall: 0.8874
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3511 - recall: 0.9044 - val_loss: 0.1699 - val_recall: 0.8874
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3444 - recall: 0.9024 - val_loss: 0.1694 - val_recall: 0.8874
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3283 - recall: 0.9088 - val_loss: 0.1961 - val_recall: 0.8874
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3358 - recall: 0.9073 - val_loss: 0.1749 - val_recall: 0.8919
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3412 - recall: 0.8935 - val_loss: 0.1635 - val_recall: 0.8874
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3268 - recall: 0.9082 - val_loss: 0.1578 - val_recall: 0.8874
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3424 - recall: 0.9088 - val_loss: 0.1650 - val_recall: 0.8874
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3041 - recall: 0.9200 - val_loss: 0.1584 - val_recall: 0.8874
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3453 - recall: 0.8925 - val_loss: 0.1617 - val_recall: 0.8874
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3257 - recall: 0.9083 - val_loss: 0.1633 - val_recall: 0.8829
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3062 - recall: 0.9085 - val_loss: 0.1552 - val_recall: 0.8829
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3075 - recall: 0.9164 - val_loss: 0.1710 - val_recall: 0.8829
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3256 - recall: 0.9125 - val_loss: 0.1580 - val_recall: 0.8919
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  31.806586503982544
In [ ]:
model_3.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout (Dropout)               │ (None, 64)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_1 (Dropout)             │ (None, 32)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 14,213 (55.52 KB)
 Trainable params: 4,737 (18.50 KB)
 Non-trainable params: 0 (0.00 B)
 Optimizer params: 9,476 (37.02 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_3_train_perf=model_performance_classification(model_3,X_train,y_train)
model_3_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.990688 0.990688 0.990692 0.99069
In [ ]:
model_3_val_perf=model_performance_classification(model_3,X_val,y_val)
model_3_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.98925 0.98925 0.989148 0.989192

Both recall have improved to 99% though the loss plot functions are still non-smooth

Model 4¶

Regularize with Batch Normalization

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_4 = Sequential()
model_4.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_4.add(BatchNormalization())
model_4.add(Dense(32,activation="relu"))
model_4.add(BatchNormalization())
model_4.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam(learning_rate=0.0001)
model_4.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_4.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.6216 - recall: 0.6038 - val_loss: 0.6390 - val_recall: 0.9189
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.9238 - recall: 0.9173 - val_loss: 0.5276 - val_recall: 0.9189
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.7494 - recall: 0.9252 - val_loss: 0.4343 - val_recall: 0.9054
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6397 - recall: 0.9190 - val_loss: 0.3623 - val_recall: 0.9009
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5604 - recall: 0.9164 - val_loss: 0.3077 - val_recall: 0.8874
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 4ms/step - loss: 0.5020 - recall: 0.9153 - val_loss: 0.2650 - val_recall: 0.8829
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.4585 - recall: 0.9153 - val_loss: 0.2344 - val_recall: 0.8784
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4245 - recall: 0.9137 - val_loss: 0.2105 - val_recall: 0.8739
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3989 - recall: 0.9154 - val_loss: 0.1928 - val_recall: 0.8739
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3781 - recall: 0.9156 - val_loss: 0.1788 - val_recall: 0.8739
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3608 - recall: 0.9152 - val_loss: 0.1677 - val_recall: 0.8739
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3454 - recall: 0.9152 - val_loss: 0.1598 - val_recall: 0.8694
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3326 - recall: 0.9152 - val_loss: 0.1518 - val_recall: 0.8694
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3207 - recall: 0.9178 - val_loss: 0.1459 - val_recall: 0.8649
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3096 - recall: 0.9178 - val_loss: 0.1414 - val_recall: 0.8649
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.2999 - recall: 0.9200 - val_loss: 0.1369 - val_recall: 0.8694
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2898 - recall: 0.9204 - val_loss: 0.1318 - val_recall: 0.8694
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2779 - recall: 0.9232 - val_loss: 0.1291 - val_recall: 0.8649
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2681 - recall: 0.9241 - val_loss: 0.1261 - val_recall: 0.8649
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2599 - recall: 0.9283 - val_loss: 0.1232 - val_recall: 0.8649
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2525 - recall: 0.9283 - val_loss: 0.1206 - val_recall: 0.8649
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2442 - recall: 0.9281 - val_loss: 0.1185 - val_recall: 0.8649
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2372 - recall: 0.9283 - val_loss: 0.1159 - val_recall: 0.8649
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2309 - recall: 0.9330 - val_loss: 0.1144 - val_recall: 0.8649
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2244 - recall: 0.9343 - val_loss: 0.1124 - val_recall: 0.8649
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.2188 - recall: 0.9351 - val_loss: 0.1106 - val_recall: 0.8649
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2128 - recall: 0.9356 - val_loss: 0.1086 - val_recall: 0.8649
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2068 - recall: 0.9404 - val_loss: 0.1072 - val_recall: 0.8649
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2019 - recall: 0.9427 - val_loss: 0.1052 - val_recall: 0.8649
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1962 - recall: 0.9452 - val_loss: 0.1041 - val_recall: 0.8649
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  36.642983198165894
In [ ]:
model_4.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ batch_normalization             │ (None, 64)             │           256 │
│ (BatchNormalization)            │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ batch_normalization_1           │ (None, 32)             │           128 │
│ (BatchNormalization)            │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 14,981 (58.52 KB)
 Trainable params: 4,929 (19.25 KB)
 Non-trainable params: 192 (768.00 B)
 Optimizer params: 9,860 (38.52 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_4_train_perf=model_performance_classification(model_4,X_train,y_train)
model_4_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 723us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.98625 0.98625 0.986826 0.986475
In [ ]:
model_4_val_perf=model_performance_classification(model_4,X_val,y_val)
model_4_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 926us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.98275 0.98275 0.983112 0.982911

Recall in both cases has reduced to 98%. The loss plot has smoothen and gap between train and validation significantly reduced.

Model 5¶

Using both Batchnormalization and dropout

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_5 = Sequential()
model_5.add(Dense(64,activation="relu",input_dim = X_train.shape[1]))
model_5.add(BatchNormalization())
model_5.add(Dropout(0.4))
model_5.add(Dense(32,activation="relu"))
model_5.add(BatchNormalization())
model_5.add(Dropout(0.2))
model_5.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam()
model_5.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_5.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.1307 - recall: 0.7977 - val_loss: 0.2494 - val_recall: 0.8784
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6416 - recall: 0.8748 - val_loss: 0.1924 - val_recall: 0.8739
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5165 - recall: 0.8909 - val_loss: 0.1566 - val_recall: 0.8874
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5173 - recall: 0.8844 - val_loss: 0.1511 - val_recall: 0.8874
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4706 - recall: 0.8970 - val_loss: 0.1374 - val_recall: 0.8829
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4742 - recall: 0.9045 - val_loss: 0.1340 - val_recall: 0.8829
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4519 - recall: 0.9019 - val_loss: 0.1296 - val_recall: 0.8829
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4544 - recall: 0.8974 - val_loss: 0.1223 - val_recall: 0.8829
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4184 - recall: 0.9075 - val_loss: 0.1156 - val_recall: 0.8829
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4317 - recall: 0.8989 - val_loss: 0.1194 - val_recall: 0.8919
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4011 - recall: 0.9074 - val_loss: 0.1189 - val_recall: 0.8874
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4134 - recall: 0.9116 - val_loss: 0.1160 - val_recall: 0.8784
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3797 - recall: 0.9150 - val_loss: 0.1171 - val_recall: 0.8874
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3886 - recall: 0.9014 - val_loss: 0.1130 - val_recall: 0.8964
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3728 - recall: 0.9109 - val_loss: 0.1111 - val_recall: 0.8874
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3694 - recall: 0.9051 - val_loss: 0.1073 - val_recall: 0.8874
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3878 - recall: 0.9084 - val_loss: 0.1064 - val_recall: 0.8874
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3711 - recall: 0.9168 - val_loss: 0.1062 - val_recall: 0.8874
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3599 - recall: 0.9109 - val_loss: 0.1052 - val_recall: 0.8829
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3674 - recall: 0.9099 - val_loss: 0.1010 - val_recall: 0.8829
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3794 - recall: 0.9046 - val_loss: 0.1014 - val_recall: 0.8829
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3483 - recall: 0.9132 - val_loss: 0.1043 - val_recall: 0.8919
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3609 - recall: 0.9163 - val_loss: 0.1026 - val_recall: 0.8829
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3464 - recall: 0.9084 - val_loss: 0.1021 - val_recall: 0.8874
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3621 - recall: 0.9148 - val_loss: 0.1082 - val_recall: 0.8964
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3311 - recall: 0.9143 - val_loss: 0.1051 - val_recall: 0.8919
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3281 - recall: 0.9236 - val_loss: 0.1040 - val_recall: 0.8919
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - loss: 0.3419 - recall: 0.9151 - val_loss: 0.0971 - val_recall: 0.8919
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3251 - recall: 0.9160 - val_loss: 0.1006 - val_recall: 0.8919
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3252 - recall: 0.9173 - val_loss: 0.0961 - val_recall: 0.8874
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  36.77445697784424
In [ ]:
model_5.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ batch_normalization             │ (None, 64)             │           256 │
│ (BatchNormalization)            │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout (Dropout)               │ (None, 64)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ batch_normalization_1           │ (None, 32)             │           128 │
│ (BatchNormalization)            │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_1 (Dropout)             │ (None, 32)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 14,981 (58.52 KB)
 Trainable params: 4,929 (19.25 KB)
 Non-trainable params: 192 (768.00 B)
 Optimizer params: 9,860 (38.52 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_5_train_perf=model_performance_classification(model_5,X_train,y_train)
model_5_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 993us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.989375 0.989375 0.989473 0.98942
In [ ]:
model_5_val_perf=model_performance_classification(model_5,X_val,y_val)
model_5_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.9875 0.9875 0.9875 0.9875

Both perfomance in regard to recall are at 98%. The loss plots are non-smooth

Model 6¶

Using weight initialization and dropout

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_6 = Sequential()
model_6.add(Dense(64,activation="relu",input_dim = X_train.shape[1],kernel_initializer='he_normal'))
model_6.add(Dropout(0.4))
model_6.add(Dense(32,activation="relu",kernel_initializer='he_normal'))
model_6.add(Dropout(0.2))
model_6.add(Dense(1,activation = 'sigmoid'))
#compiling the model With Adam as optimiser
optimizer = keras.optimizers.Adam()
model_6.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_6.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 1.9393 - recall: 0.6566 - val_loss: 0.3760 - val_recall: 0.8829
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.8441 - recall: 0.8062 - val_loss: 0.3082 - val_recall: 0.8874
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.6197 - recall: 0.8560 - val_loss: 0.2725 - val_recall: 0.8829
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5885 - recall: 0.8529 - val_loss: 0.2644 - val_recall: 0.8919
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.5384 - recall: 0.8843 - val_loss: 0.2571 - val_recall: 0.8784
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4891 - recall: 0.8885 - val_loss: 0.2639 - val_recall: 0.8784
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4619 - recall: 0.8862 - val_loss: 0.2133 - val_recall: 0.8784
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.4493 - recall: 0.8742 - val_loss: 0.2134 - val_recall: 0.8874
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4468 - recall: 0.8824 - val_loss: 0.2146 - val_recall: 0.8919
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4250 - recall: 0.8874 - val_loss: 0.2157 - val_recall: 0.8829
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4205 - recall: 0.8945 - val_loss: 0.1953 - val_recall: 0.8829
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - loss: 0.3911 - recall: 0.9025 - val_loss: 0.1974 - val_recall: 0.8919
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3925 - recall: 0.8878 - val_loss: 0.1863 - val_recall: 0.8874
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.4054 - recall: 0.8866 - val_loss: 0.1832 - val_recall: 0.9009
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3833 - recall: 0.8860 - val_loss: 0.1866 - val_recall: 0.9009
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3544 - recall: 0.9095 - val_loss: 0.1765 - val_recall: 0.8964
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3699 - recall: 0.8974 - val_loss: 0.1845 - val_recall: 0.8964
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3412 - recall: 0.9120 - val_loss: 0.1730 - val_recall: 0.8919
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3595 - recall: 0.9030 - val_loss: 0.1667 - val_recall: 0.8874
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3499 - recall: 0.9066 - val_loss: 0.1670 - val_recall: 0.9009
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3585 - recall: 0.9036 - val_loss: 0.1723 - val_recall: 0.8919
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3324 - recall: 0.9080 - val_loss: 0.1634 - val_recall: 0.8874
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3364 - recall: 0.9035 - val_loss: 0.1696 - val_recall: 0.8964
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3326 - recall: 0.9075 - val_loss: 0.1801 - val_recall: 0.9009
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3219 - recall: 0.9076 - val_loss: 0.1701 - val_recall: 0.8919
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3239 - recall: 0.8982 - val_loss: 0.1535 - val_recall: 0.8964
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3150 - recall: 0.9148 - val_loss: 0.1679 - val_recall: 0.8919
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.3318 - recall: 0.9097 - val_loss: 0.1603 - val_recall: 0.8874
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3095 - recall: 0.9121 - val_loss: 0.1662 - val_recall: 0.8919
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3266 - recall: 0.9113 - val_loss: 0.1726 - val_recall: 0.8964
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  35.58531212806702
In [ ]:
model_6.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout (Dropout)               │ (None, 64)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_1 (Dropout)             │ (None, 32)             │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 14,213 (55.52 KB)
 Trainable params: 4,737 (18.50 KB)
 Non-trainable params: 0 (0.00 B)
 Optimizer params: 9,476 (37.02 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
model_6_train_perf=model_performance_classification(model_6,X_train,y_train)
model_6_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 682us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.990938 0.990938 0.990923 0.99093
In [ ]:
model_6_val_perf=model_performance_classification(model_6,X_val,y_val)
model_6_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.9885 0.9885 0.9885 0.9885

Both recals are around 99% and loss plots are smoothening

Model 7¶

Weight initialization and Batch Normalization

In [ ]:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_7 = Sequential()
model_7.add(Dense(64,activation="relu",input_dim = X_train.shape[1],kernel_initializer='he_normal'))
model_7.add(BatchNormalization())
model_7.add(Dense(32,activation="relu",kernel_initializer='he_normal'))
model_7.add(BatchNormalization())
model_7.add(Dense(1,activation = 'sigmoid'))
#compiling the model with Adam as optimiser
optimizer = keras.optimizers.Adam()
model_7.compile(optimizer =optimizer,loss = 'binary_crossentropy',metrics = ['recall'])
start=time.time()
history=model_7.fit(X_train,y_train,validation_data=(X_val,y_val),epochs=30,batch_size=32,class_weight=cw_dict)
end=time.time()
Epoch 1/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 4s 4ms/step - loss: 0.8269 - recall: 0.8495 - val_loss: 0.2436 - val_recall: 0.8874
Epoch 2/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 3s 4ms/step - loss: 0.4502 - recall: 0.9085 - val_loss: 0.1806 - val_recall: 0.8874
Epoch 3/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3810 - recall: 0.9101 - val_loss: 0.1518 - val_recall: 0.8919
Epoch 4/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3420 - recall: 0.9193 - val_loss: 0.1358 - val_recall: 0.8829
Epoch 5/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.3069 - recall: 0.9251 - val_loss: 0.1351 - val_recall: 0.8919
Epoch 6/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2787 - recall: 0.9344 - val_loss: 0.1294 - val_recall: 0.8919
Epoch 7/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2627 - recall: 0.9336 - val_loss: 0.1273 - val_recall: 0.8919
Epoch 8/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2485 - recall: 0.9327 - val_loss: 0.1219 - val_recall: 0.8964
Epoch 9/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2287 - recall: 0.9410 - val_loss: 0.1196 - val_recall: 0.8919
Epoch 10/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2274 - recall: 0.9392 - val_loss: 0.1166 - val_recall: 0.8919
Epoch 11/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.2068 - recall: 0.9497 - val_loss: 0.1122 - val_recall: 0.8874
Epoch 12/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1832 - recall: 0.9531 - val_loss: 0.1144 - val_recall: 0.8874
Epoch 13/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1739 - recall: 0.9573 - val_loss: 0.1041 - val_recall: 0.8874
Epoch 14/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1623 - recall: 0.9588 - val_loss: 0.1029 - val_recall: 0.8694
Epoch 15/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1551 - recall: 0.9604 - val_loss: 0.0987 - val_recall: 0.8739
Epoch 16/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1395 - recall: 0.9626 - val_loss: 0.1025 - val_recall: 0.8874
Epoch 17/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1260 - recall: 0.9639 - val_loss: 0.0908 - val_recall: 0.8694
Epoch 18/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1334 - recall: 0.9629 - val_loss: 0.0908 - val_recall: 0.8694
Epoch 19/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1130 - recall: 0.9697 - val_loss: 0.1303 - val_recall: 0.8784
Epoch 20/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.1184 - recall: 0.9680 - val_loss: 0.0834 - val_recall: 0.8694
Epoch 21/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0891 - recall: 0.9777 - val_loss: 0.0903 - val_recall: 0.8649
Epoch 22/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 2s 2ms/step - loss: 0.0900 - recall: 0.9766 - val_loss: 0.0883 - val_recall: 0.8649
Epoch 23/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0859 - recall: 0.9758 - val_loss: 0.0891 - val_recall: 0.8694
Epoch 24/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0808 - recall: 0.9809 - val_loss: 0.0867 - val_recall: 0.8423
Epoch 25/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0684 - recall: 0.9847 - val_loss: 0.0953 - val_recall: 0.8649
Epoch 26/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0555 - recall: 0.9896 - val_loss: 0.0862 - val_recall: 0.8514
Epoch 27/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0625 - recall: 0.9825 - val_loss: 0.1080 - val_recall: 0.8468
Epoch 28/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0870 - recall: 0.9816 - val_loss: 0.0921 - val_recall: 0.8694
Epoch 29/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0701 - recall: 0.9832 - val_loss: 0.0928 - val_recall: 0.8739
Epoch 30/30
500/500 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - loss: 0.0501 - recall: 0.9902 - val_loss: 0.0931 - val_recall: 0.8694
In [ ]:
print("Time taken in seconds ",end-start)
Time taken in seconds  38.600029945373535
In [ ]:
model_7.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 64)             │         2,624 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ batch_normalization             │ (None, 64)             │           256 │
│ (BatchNormalization)            │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 32)             │         2,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ batch_normalization_1           │ (None, 32)             │           128 │
│ (BatchNormalization)            │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 14,981 (58.52 KB)
 Trainable params: 4,929 (19.25 KB)
 Non-trainable params: 192 (768.00 B)
 Optimizer params: 9,860 (38.52 KB)
In [ ]:
plot(history,"loss")
No description has been provided for this image
In [ ]:
#training performance
model_7_train_perf=model_performance_classification(model_7,X_train,y_train)
model_7_train_perf
500/500 ━━━━━━━━━━━━━━━━━━━━ 0s 785us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.986812 0.986812 0.987221 0.986979
In [ ]:
#validation performance
model_7_val_perf=model_performance_classification(model_7,X_val,y_val)
model_7_val_perf
125/125 ━━━━━━━━━━━━━━━━━━━━ 0s 902us/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.98425 0.98425 0.984428 0.984332

Model Performance Comparison and Final Model Selection¶

Now, in order to select the final model, we will compare the performances of all the models for the training and validation sets.

In [ ]:
#training performance comparision
model_train_perf_comp=pd.concat([model_0_train_perf.T,model_1_train_perf.T,model_2_train_perf.T,
                                 model_3_train_perf.T,model_4_train_perf.T, model_5_train_perf.T,
                                 model_6_train_perf.T,model_7_train_perf.T,
    ],
      axis=1
    )
model_train_perf_comp.columns=[
    "Neural Network (SGD, No Regularization)",
    "Neural Network (SGD with Momentum, No Regularization)",
    "Neural Network (Adam , No Regularization)",
    "Neural Network (Adam, dropout [0.4,0.2])",
    "Neural Network (Adam, Batch Normalization)",
    "Neural Network (Adam,dropout [0.4,0.2], Batch Normalization)",
    "Neural Network (Adam,dropout [0.4,0.2] ,He initialization)",
    "Neural Network (Adam,He initialization, Batch Normalization)"
                               ]
print("Training perfomance comparision:")

model_train_perf_comp
Training perfomance comparision:
Out[ ]:
Neural Network (SGD, No Regularization) Neural Network (SGD with Momentum, No Regularization) Neural Network (Adam , No Regularization) Neural Network (Adam, dropout [0.4,0.2]) Neural Network (Adam, Batch Normalization) Neural Network (Adam,dropout [0.4,0.2], Batch Normalization) Neural Network (Adam,dropout [0.4,0.2] ,He initialization) Neural Network (Adam,He initialization, Batch Normalization)
Accuracy 0.981750 0.993437 0.987812 0.990688 0.986250 0.989375 0.990938 0.986812
Recall 0.981750 0.993437 0.987812 0.990688 0.986250 0.989375 0.990938 0.986812
Precision 0.984608 0.993372 0.989453 0.990692 0.986826 0.989473 0.990923 0.987221
F1 Score 0.982628 0.993393 0.988276 0.990690 0.986475 0.989420 0.990930 0.986979
In [ ]:
#validation perfomance comparision
model_val_perf_comp=pd.concat([model_0_val_perf.T,model_1_val_perf.T,model_2_val_perf.T, model_3_val_perf.T,
                               model_4_val_perf.T,model_5_val_perf.T,model_6_val_perf.T,model_7_val_perf.T,
                               ],axis=1
                              )
model_val_perf_comp.columns=[
    "Neural Network (SGD, No Regularization)",
    "Neural Network (SGD with Momentum, No Regularization)",
    "Neural Network (Adam , No Regularization)",
    "Neural Network (Adam, dropout [0.4,0.2])",
    "Neural Network (Adam, Batch Normalization)",
    "Neural Network (Adam,dropout [0.4,0.2], Batch Normalization)",
    "Neural Network (Adam,dropout [0.4,0.2] ,He initialization)",
    "Neural Network (Adam,He initialization, Batch Normalization)"
                             ]
print("Validation perfomance comparision:")

model_val_perf_comp
Validation perfomance comparision:
Out[ ]:
Neural Network (SGD, No Regularization) Neural Network (SGD with Momentum, No Regularization) Neural Network (Adam , No Regularization) Neural Network (Adam, dropout [0.4,0.2]) Neural Network (Adam, Batch Normalization) Neural Network (Adam,dropout [0.4,0.2], Batch Normalization) Neural Network (Adam,dropout [0.4,0.2] ,He initialization) Neural Network (Adam,He initialization, Batch Normalization)
Accuracy 0.978000 0.988000 0.979250 0.989250 0.982750 0.9875 0.9885 0.984250
Recall 0.978000 0.988000 0.979250 0.989250 0.982750 0.9875 0.9885 0.984250
Precision 0.980238 0.987951 0.980815 0.989148 0.983112 0.9875 0.9885 0.984428
F1 Score 0.978812 0.987974 0.979846 0.989192 0.982911 0.9875 0.9885 0.984332

Neural network model 3 with Adam as optimizer and dropout[0.4,0.2] as regularization is chosen since it has the highest recall hence best in addressing false negative associated with high replacement cost

Now, let's check the performance of the final model on the test set.

In [ ]:
#Perfomance of model_3 on test
model_3_test_perf=model_performance_classification(model_3,X_test,y_test)
model_3_test_perf
157/157 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step
Out[ ]:
Accuracy Recall Precision F1 Score
0 0.9882 0.9882 0.987997 0.988069

The perfomance is within range with a recall of 98.82 percent, suggesting the model is accurately classifying instances with minimal false negatives.

Actionable Insights and Recommendations¶

  • We have build a model that can be used to predict failures so that the generators could be repaired before failing/breaking to reduce the overall maintenance cost.

  • The model is accurately classifying instances with minimal false negatives which has the highest maintenance cost interms of replacement cost hence gradually reducing maintanance cost.

  • The company can deploy the final model from this exercise to identify with a reasonable degree of accuracy whether failures will occur or not, and this process seems to be easier and more time-efficient than other methods

  • The model could further be continously trained on large data to enhance it performance.